System and method for providing data manipulation as a web service

ABSTRACT

A method of transforming data includes receiving information defining a transformation of an input data object to an output data object and generating, based on the received information, a script operable when executed to implement the defined transformation. The script is then stored. The method also includes receiving a service request from a remote client that requests a particular transformation and identifies a request data object and identifying a script that performs the requested transformation. Additionally, the method includes generating a response data object by executing the identified script on the request data object and transmitting the response data object to the remote client.

RELATED APPLICATIONS

This application claims the priority under 35 U.S.C. §119 of provisionalapplication Ser. No. 60/659,264 filed Mar. 7, 2005, which isincorporated by reference.

TECHNICAL FIELD OF THE INVENTION

This disclosure relates generally to the field of data processing and,more particularly, to a system and method for manipulating data.

BACKGROUND OF THE INVENTION

In the rapidly-evolving competitive marketplace, data is among anorganization's most valuable assets. Business success demands access todata and information, and the ability to quickly and seamlesslydistribute data throughout the enterprise to support business processrequirements. Organizations must extract, refine, manipulate, transform,integrate and distribute data in formats suitable for strategicdecision-making. This poses a unique challenge in heterogeneousenvironments, where data is housed on disparate platforms in any numberof different formats and used in many different contexts.

SUMMARY OF THE INVENTION

In accordance with the present invention, the disadvantages and problemsassociated with data processing have been substantially reduced oreliminated. In particular, methods and systems for transforming data aredisclosed that provide a flexible, robust manner for providing datatransformation functionality as a web service.

In accordance with one embodiment of the present invention, a method oftransforming data includes receiving information defining atransformation of an input data object to an output data object andgenerating, based on the received information, a script operable whenexecuted to implement the defined transformation. The script is thenstored. The method also includes receiving a service request from aremote client that requests a particular transformation and identifies arequest data object and identifying a script that performs the requestedtransformation. Additionally, the method includes generating a responsedata object by executing the identified script on the request dataobject and transmitting the response data object to the remote client.

In accordance with another embodiment of the present invention, a methodof transforming data includes receiving information defining atransformation of an input data object to an output data object andgenerating, based on the received information, a script operable whenexecuted to implement the defined transformation based on the receivedinformation. The script is then stored. The method also includesreceiving, from a remote client, a service request that requests thedefined transformation and identifies a request data object and arequested execution time. Additionally, the method includes generating aresponse data object by executing the script on the request data objectat a time determined based on the requested execution time and storingthe response data object in memory.

Some embodiments of the present invention provide numerous technicaladvantages. Other embodiments may realize some, none, or all of theseadvantages. For example, particular embodiments may provide a dataextraction, transformation, and load tool that features a flexible,easy-to-use, and comprehensive application-development environment.Particular embodiments may also reduce and/or eliminate the programmingcomplexities of extracting, transforming, and loading data fromdisparate sources and targets and eliminate a need for users to learnXML programming or database-specific API's. Embodiments of the inventionmay facilitate seamless extraction and integration of data from and toAS/400, DB2, DB2 MVS, DBASE, flat files, COBOL files, Lotus Notes,Microsoft ODBC, Microsoft SQL Server, Oracle, Sybase, Microsoft Access,CA Ingres and UDB.

In particular embodiments, some features provide the ability to processand output a wide variety of different types of input files and outputfiles with significant flexibility in how the data may be transformed.As one example, particular embodiments of the described system may becapable of accepting input files in an XML format, transforming thedata, and outputting the transformed data in one or more database tablesor flat files. Similarly, particular embodiments may be capable ofaccepting input database tables or flat files, transforming the datacontained in these files, and outputting the transformed data in onemore XML files. As another example, particular embodiments of thedescribed system may be capable of reading and transforming documentshaving a variable number of instances of a particular data object. As aresult, the described system and methods provide a powerful, robust datatransformation solution

Other technical advantages of the present invention will be readilyapparent to one skilled in the art from the following figures,descriptions, and claims. Moreover, while specific advantages have beenenumerated above, various embodiments may include all, some, or none ofthe enumerated advantages.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a system for data manipulationaccording to one embodiment of the invention;

FIG. 2 is a block diagram illustrating a mapper for data manipulationaccording to one embodiment of the invention;

FIGS. 3A and 3B are example screen shots illustrating some functionalityof an XML Object Definition of the mapper of FIG. 2;

FIG. 4 is an example screen shot illustrating some functionality of anexample Mapping module of the mapper of FIG. 2;

FIGS. 5A-5C illustrate an example script generated by a particularembodiment of the data manipulation system;

FIG. 6 is a block diagram illustrating an XML Interface for datamanipulation according to one embodiment of the invention;

FIG. 7A is a flowchart illustrating an example method of executing ascript to perform a first transformation of data from a database sourcefile to an XML target file according to one embodiment of the invention;

FIG. 7B is an example output of the example method of FIG. 7A accordingto one embodiment of the invention;

FIG. 8 is a flowchart illustrating an example method of executing ascript to perform a second transformation of data from an XML sourcefile to a database target file according to one embodiment of theinvention;

FIG. 9 shows a particular embodiment of a data transformation systemcapable of providing data transformation functionality to remote clientsas a web service; and

FIG. 10 show a particular embodiment of a data transformation systemcapable of utilizing web services offered by remote web servers as partof data transformation functionality supported by the system;

FIG. 11 is an example screen shot illustrating some functionality of anexample Mapping module that may be utilized by particular embodiments ofthe system shown in FIG. 10; and

FIGS. 12A and 12B show an example script that may be generated byparticular embodiments of the system illustrated in FIG. 10.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

FIG. 1 is a block diagram illustrating a system 100 for datamanipulation according to one embodiment of the present invention.Generally, system 100 includes a graphical data movement tool that isreferred to herein as Advantage Data Transformer (“ADT”) 101. Someembodiments of the invention facilitate extensible Markup Language(“XML”) functionality with an ability to use XML document files aseither sources or targets and transform data between XML format anddatabase format, flat file format, or any other appropriate formats.Various embodiments of system 100 and ADT 101 are described below inconjunction with FIGS. 1 through 8.

In the illustrated embodiment, ADT 101 includes a mapper module 102, ascript manager 104, a server 106, interfaces 108, XML files 110,database tables or files 112, and an internal database 114. The presentinvention contemplates more, fewer, or different components associatedwith ADT 101 than those illustrated in FIG. 1. In addition, any of theelements or various functions of the elements of ADT 101 may be suitablydistributed among one or more computers, servers, computer systems, andnetworks in any suitable location or locations. As such, any suitablenumber of processors may be associated with, and perform the functionsof, ADT 101.

Mapper module 102 includes any suitable hardware, software, firmware, orcombination thereof operable to receive a first data definition (or fileformat) of a source file, receive a second data definition (or fileformat) of a target file, and automatically generate a script 115 torepresent a movement of data from the source file to the target file.For the purposes of this description and the claims that follow, theterm “file” may be used to refer to a collection of data structured inany suitable manner. As a result, a “file” may contain data stored in ahierarchical structure, in a relational form, or any other appropriatemanner, and a “file” may represent all or a portion of an XML file, arelational database, a flat file, or any other appropriate collection ofdata. Furthermore, as used herein, the term “automatically” generallymeans that the appropriate processing is substantially performed bymapper module 102. However, in particular embodiments, use of the term“automatically” may contemplate appropriate user interaction with mappermodule 102. As described in greater detail below, mapper module 102includes one or more suitable graphical user interfaces (“GUIs”) and,among other functions, allows a user to design data formats, scan dataformats from existing definitions, edit existing data formats, anddesign data transformation programs via drag-and-drop functionality.Further details of mapper module 102 are described below in conjunctionwith FIG. 2.

Script manager 104 includes any suitable hardware, software, firmware,or combination thereof operable to manage scripts 115 generated bymapper module 102. This may include storing scripts 115 in internaldatabase 114 or other suitable storage locations, and may includescheduling scripts 115 for execution by server 106. Script manager 104may also provide database connection information to source tables andtarget tables through suitable database profiles. In particularembodiments, database profiles may specify a particular interface,server name, database name, user ID, and password for a particularprogram.

Server 106 includes any suitable hardware, software, firmware, orcombination thereof operable to execute scripts 115 when directed byscript manager 104. The transformation of data formats takes place inscripts 115 when executed by server 106. In other words, server 106 mayperform the data movements from one or more source files to one or moretarget files. As used herein, Other functionalities performed by server106 are contemplated by the present invention.

In one embodiment, the flow between script manager 104 and server 106 isas follows: script manager 104 puts a script run request in a queue ininternal database 114 when a user selects a script to run. A schedulerfunction within server 106 picks up the run request and verifies thescript is valid to run. Server 106 then starts an interpreter functionto run the relevant script. The interpreter pulls the compiled scriptfrom internal database 114 and starts interpreting (i.e., running) thescript. The interpreter loads interfaces 108 during script execution.The interfaces 108 access files based on the script. Messages from thescript get logged in internal database 114. The scheduler logs scriptreturn code in internal database 114, and script manager 104 inspectsinternal database 114 logs for script messages and return codes. Scriptmanager 104 can view the execution and message logs from internaldatabase 114 to report on status and completion of script execution.

Interfaces 108, in the illustrated embodiment, include an XML interface600 and a database interface 116. However, the present inventioncontemplates other suitable interfaces. Interfaces 108 include anysuitable hardware, software, firmware, or combination thereof operableto load and store a particular format of data when called by server 106in accordance with scripts 115. Interfaces 108 may couple to sourcefiles and target files during execution of scripts 115. For example, inthe embodiment illustrated in FIG. 1, XML interface 600 is coupled toXML files 110 and database interface 116 is coupled to database tables112. XML files 110 and database tables 112 are representative of variousdata stored in various file formats and may be associated with anysuitable platform including, but not limited to, Windows NT, Windows2000, Windows 2003, Windows XP, Linux, AIX, HP-UX, and Sun Solaris. Thepresent invention contemplates interfaces 108 having other suitablefunctionalities. Further details of XML interface 600 are describedbelow in conjunction with FIG. 6.

FIG. 2 is a block diagram illustrating mapper module 102 according toone embodiment of the invention. In the illustrated embodiment, mappermodule 102 includes one or more editors 208 and Script Generation module204. Mapper module 102 also includes one or more data definitions 200, aMapping module 202, and one or more scanners 206 each associated withone or more data formats that mapper module 102 is capable of receivingand/or outputting. For example, in the illustrated example mapper module102 includes or stores an XML Object Definition 200 a, and an XMLScanner 206 a for supporting functionality associated with thetransformation of XML data. Similarly, the illustrated example alsoincludes additional data definitions 200 (e.g. relational tabledefinition 200 b and flat file record definition 200 c), and scanners206 associated with DBMS files and flat files respectively. AlthoughFIG. 2 illustrates a particular embodiment of mapper module 102 thatincludes particular components capable of supporting a number ofspecific data formats, alternative embodiments may include mappermodules 102 capable of supporting any appropriate number and types ofdata formats.

Data Definitions 200, in particular embodiments, are each operable toreceive a data definition (or file format) of a source file and/or atarget file. This may be accomplished in any suitable manner andparticular embodiments may allow a user to define or design such dataformat. Two ways to define a particular data format may be via scanningwith an appropriate Scanner 206 or by manual entry with the help of anappropriate editor 208. Pre-existing data definitions (i.e., fileformats) may also be stored in internal database 114.

Scanners 206 are each operable to automatically generate a data formatfrom an existing definition that contains the associated data format.One example of such data format scanning is described in U.S. patentapplication Ser. No. 11/074,502, which is herein incorporated byreference. The manual definition of an example XML document file formatis shown and described below in conjunction with FIGS. 3A and 3B. Manualdefinitions for files of other formats maybe entered in a similarfashion with appropriate modifications using an editor 208 correspondingto the relevant file format.

Mapping module 202 is operable to allow a user to design atransformation program to transform data of a particular format (e.g.XML) via mappings from one or more source files to one or more targetfiles. This may be accomplished via a GUI having a program palette inwhich a user is allowed to drag and drop source data definitions into atarget data definition therein in order to perform the desiredconnection. Such a program palette is shown and described below inconjunction with FIG. 4. These graphical mappings by a user represents adesired movement of data from the source files to the target file.

Script Generation module 204 is operable to automatically convert themappings captured by Mapping module 202 into a script to represent themovement of data from the source files to the target file. An examplescript is shown and described below in conjunction with FIGS. 5A-5C.

FIGS. 3A and 3B are example screen shots illustrating some functionalityof XML Object Definition 200 according to a particular embodiment.Although the description below focuses for purposes of illustration onthe transformation of XML data, as noted above, mapper module 102 may beconfigured to utilize any appropriate form of data for input and outputfiles. Referring first to FIG. 3A, a “Create New XML Object” dialog 300is illustrated. Dialog 300 allows a user to create a new XML object. Theuser may select the target folder where the XML object is to be createdby using a window 302. A browser tree may be associated with window 302from which a user may select a desired XML folder. The user may enter aname for the new XML object into window 304. A list of existing XMLobjects in the selected folder may be displayed in a window 306 to aidthe user when defining the name of a new XML object. The XML object'sfully qualified path and name may be also shown in a window 308. Othersuitable windows may be associated with dialog 300, such as a statuswindow 310 and a version window 312. Once all the desired information isentered into dialog 300, a user clicks on a Finish button 314 to createthe XML object. XML Object Definition 200 then launches an “XML ObjectDefinition” dialog 350, as illustrated in FIG. 3B below.

XML Object Definition dialog 350 allows a user to define elements,attributes, namespaces, comments, and/or other markup languagecomponents (generically referred to here as “data components”) thatdefine the layout of an XML object that the user may later use as asource or target on a program palette. The XML Object definition definesthe layout of the object and controls how the data is read/written whenused in a program. In the illustrated embodiment, dialog 350 illustratesan XML file format 351 in a window 352 for the PersonCars_Document filethat is shown in the Create New XML Object dialog 300 above. The XMLObject Definition dialog 350 allows a user to create and/or modify XMLdata components of an XML object that include elements, repeatingelements, attributes, namespaces, and comments. An icon with aparticular letter or symbol may be displayed for each component in XMLfile format 351. In the illustrated embodiment, an “E” is used toillustrate an element type, an “R” is used to illustrate a repeatingelement type, an “A” is used to illustrate an attribute of an element,an “N” is used to illustrate a namespace, and an “!” is used toillustrate a comment. Nonetheless, particular embodiments of XML ObjectDefinition dialog 350 may use other appropriate designations.

Component information 353 describing characteristics of a particularcomponent in XML file format 351 is displayed in a component informationwindow 354 as a user moves a cursor over a particular component or whena user selects a single item in XML file format 351. Componentinformation window 354 shows component information, such as the type,name, value, namespace prefix, namespace URI, use CDATA, and whether ornot it is a repeating element. Dialog 350 may have a number of suitableoperations 356 associated with it. A “New” operation 357 invokes an XMLcomponent dialog to create a new XML component using the currentlyselected element or element parent. An “Edit” operation 358 invokes anXML component dialog to edit an existing XML component. In this case,the XML file format 351 may be synchronized with the updated componentdata. A “Delete” operation 359 deletes the selected component andchildren, if desired. Deleting an element may result in deleting otherXML components, such as children elements, attributes, comments, ornamespaces.

Additionally, a “Validate XML file format” operation 360 may performvalidation of a current file format 351. For example, in particularembodiments, the “Validate XML file format” operation 360 may performthe following checks for a particular file format, report theappropriate results, and select the offending component in the fileformat for further correction:

-   -   1. Verify that the namespace prefixes and URIs are correct for        their relevant scope    -   2. Verifies that the XML data component names are valid and do        not contain invalid characters.    -   3. Verifies that the XML data component names are unique for the        scope under which they are defined.    -   4. Performs special tests for element names including, for        example, determining whether second level (record) qualified        element names are unique and determining whether the qualified        names for “repeating” elements at the third level or lower are        unique.    -   5. Checks to see if an XML file format contains valid second        level elements. The definition may be invalid if it contains two        or more second level elements and has repeating elements        designated.        Alternative embodiments may utilize additional or alternative        checks to verify the data component.

A “Repeating Element” operation 361 is used to designate an element asrepeating or not repeating. The handling of repeating elements isdescribed in further detail below. “Movement” operations 362 move asingle or group of components in a particular direction to change theorder of hierarchy of data components within file format 351.

Although the above description focuses on a particular embodiment of ADT101 that supports certain functionality for “Create New XML Object”dialog 300 and XML Object Definition dialog 350, alternative embodimentsmay support any appropriate functionality for the creation anddefinition of XML objects. For example, in particular embodiments, auser may be able to drag and drop one or more data components in asingle operation. In addition, a context menu or other suitable menu maybe shown when a user right clicks on components of file format 351. Thismenu may have suitable menu items that are comparable to the operations356 discussed above.

FIG. 4 is an example screen shot 400 illustrating some functionality ofMapping module 202 according to one embodiment of the invention. Screenshot 400 includes a GUI with a program palette 402 that allows a user todesign a desired movement of data from one or more source databasetables 404 to a target data definition 406 using one or more graphicalmappings. These mappings are first facilitated by a simple dragging anddropping of data definitions into program palette 402. For example, inthe illustrated embodiment, source tables 404 a, 404 b aredragged-and-dropped into program palette 402. In addition, a target datadefinition 406 is dragged-and-dropped into program palette 402. Then theindividual “fields” from source tables 404 a, 404 b are mapped toindividual elements in target data definition 406. As indicated by thearrows in program palette 402, the “id” field in source table 404 a ismapped by the user to the “Id” element in target data definition 406,the “name” field in source table 404 a is mapped to the “Name” elementin target data definition 406, and the “address” field in source table404 a is mapped to the “Address” element in target data definition 406.In this example, a car element 411 in target data definition 406 isdesignated as a repeating element and has its own repeating elementdefinition 408. Thus, there are mappings from source table 404 b torepeating element definition 408. Any suitable mappings are contemplatedby the present invention and are controlled by the desires of the user.

A connection indicator 410 indicates that target data definition 406 andrepeating element definition 408 are related and also shows thedependency between elements and the direction of the dependency. Inaddition to showing the relationship between target data definition 406and repeating element definition 408, connection indicator 410 may alsomaintain and enforce the correct process order for the parent/childrelationships between components on program palette 402, and may enforcethe correct process order when the order is manually updated in asuitable process order dialog. More specifically, the order of scriptstatements that is produced in the resulting script uses a process orderalgorithm that uses the program palette source to target relationships(e.g., mappings, user-constraints, foreign keys, and repeating elementconnections) and produces the required DO/WHILE loops, CONNECT, SEND,LOAD, STORE, DISCONNECT, nested loops, source to target column/elementassignment statements, conditional statements, transformations and otherscript constructs. In one embodiment, a user may be prohibited fromdeleting connection indicator 410.

In particular embodiments, an expand/collapse usability feature allows auser to expand and collapse the display of target object definition 406and repeating element definition 408 on program palette 402. Thisfeature may allow a user to see target object definition 406 in a singlepalette object in the same form as shown in XML Object Definition dialog350. The collapsed view presents target object definition 406 in a formthat may help aid the user when viewing the mapping relationshipsbetween other objects on program palette 402.

In the embodiment illustrated in FIG. 4, source table 404 a is adatabase table that contains the IDs, names, and addresses of persons,and source table 404 b is a database table that contains the IDs, makes,models, and years of cars associated with those persons in source table404 a. The data in source tables 404 a, 404 b are desired to betransformed into an XML document that has a format defined by targetdata definition 406, which may have been designed using the XML ObjectDefinition 200 illustrated in FIG. 2. The example mappings in FIG. 4 areexamples that illustrate the use of program palette 402 to performgraphical mappings that correspond to a transformation of data from oneformat to another format. Any suitable mappings are contemplated by thepresent invention and transformations from any suitable format to anyother suitable format are contemplated by the present invention. Forexample, transformations may be desired from database tables to XMLfiles, XML files to database tables, XML files to other XML files,database tables to other database tables, and/or any other suitabletransformations.

Once the desired mappings are entered by a user, script generationmodule 204 may then, in response to a selection by the user,automatically convert the mappings into a script to represent themovement of data from source tables 404 to target data definition 406.An example script 500 is shown and described below in conjunction withFIGS. 5A-5C.

Thus, target data definition 406 and repeating element definition 408 onprogram palette 402 allow a user to graphically see DO WHILE loops andcorresponding LOAD/STORE units that are implicit in the transformationdefined by the user to be implemented by the generated script. Repeatingelement connections, as indicated by connection indicator 410, showcontrol sequence of execution operations and corresponding executionloops.

Mapping module 202 supports other suitable operations and/or mappinggestures for adding, deleting, and modifying data definitions defined ina transformation program. Mapping module 202 also contains specialoperations for selecting, updating, and moving objects on programpalette 402. In addition, it includes a unique “Generate Layout” featurethat arranges the palette objects for main data definition 406 andrepeating element definition 408 using non-overlapping hierarchicalrepresentation as defined in XML file format 351 (FIG. 3B). This featureis useful for automatically generating a layout that shows theparent/child relationships and hierarchy without using dialog 350 as areference.

FIGS. 5A, 5B, and 5C illustrate an example script 500 for transformingXML data according to one embodiment of the invention that is generatedby script generation module 204 (FIG. 2). As described above, inparticular embodiments, a user may define a transformation program viathe graphical mappings by dragging various source data definitions andtarget data definitions onto a program palette. In one embodiment, eachdefinition on the program palette is represented in memory as a C++object, which includes information about whether the file is a source ora target, whether the data is a table in a relational database or an XMLfile, and what columns or elements participate in the transformation.When the user maps a column or element of one source to a column orelement of a target, an in-memory C++ connection object is createdcontaining the source/target information.

During script generation, the information in the in-memory palettesource/target and connection objects is translated into data structuresthat are used to define the corresponding script code and correspondingscript structures used during the script creation process. Whereas thefirst in-memory palette and connection objects represent the appearanceof the program on the program palette, the later script data structuresrepresent the processing implied by that appearance. A list of scriptdata structures may be used to represent such granular pieces ofprocessing as CONNECTing to a database table, starting a DO WHILE loopto LOAD a row of a source table, assigning the value of one scriptvariable to another, STORing a row to a target table, or terminating aDO WHILE loop.

Finally, a number of passes are made through the array of script datastructures to write out actual script statements to define the standardscript constants, script structures to hold table column values, dataprofile names, and the actual CONNECT/DOWHILE/LOAD/IF/assignment/STORE/DISCONNECT processing statements.

As shown in FIGS. 5A-5C, example script 500 may define an array 502 anda transformation routine 504. The array 502 is an example of how an XMLfile format or data definition may be defined within the programmingcode. Whereas LOAD and STORE handlers for other interfaces, such asdatabase interface 116, may take a #DATA parameter (as shown by the lineof code at reference numerals 512) that specifies a structure withinwhich each field corresponds to a column within a database table, the#DATA parameter (as shown by the line of code at reference numeral 514)for XML LOAD and STORE handlers, according to particular embodiments,specifies an array of structures. Each of the structures in the arraycorresponds to an element, attribute, namespace, or comment specified inthe XML document. As one example, the structure may look like this:

TYPE xmlComponentDef AS STRUCTURE ( comp_name STRING, REM* Element tag,attribute name, or null comp_value STRING, REM* character value ofelement or attribute comp_type INT, REM* 0=attribute,1=element,2=namespace,3=comment comp_id INT, REM* id of the componentcomp_parent INT, REM* id of the parent element-type componentcomp_namespaceURI STRING, REM* full URI of namespace comp_NS_PrefixSTRING, REM* prefix for namespace qualification comp_IsCDATA BOOLEAN,REM* TRUE = data to be wrapped in CDATA tags comp_datatype INT, REM*datatype of element comp_IsRepeating BOOLEAN, REM* TRUE = element mayrepeat comp_level INT, REM* level of the component )CONST_comptype_attribute = 0 CONST_COMPTYPE_ATTRIBUTE = 0CONST_comptype_element = 1 CONST_COMPTYPE_ELEMENT = 1CONST_comptype_namespace = 2 CONST_COMPTYPE_NAMESPACE = 2CONST_comptype_comment = 3 CONST_COMPTYPE_COMMENT = 3

In the illustrated example, the fields are defined as follows: comp_namesimple name of element or attribute or text of comment comp_valuecharacter value of element or attribute comp_type 0 => element, 1 =>attribute, 2 => namespace component, 3 => comment comp_id a uniquenumber to identify a component; sequentially assigned starting with zerocomp_parent id of this component's parent component comp_namespaceURIthe Uniform Resource Identifier for the component's namespacecomp_NS_Prefix the prefix associated with the namespaceURI comp_IsCDATAused to indicate that the character value may contain problematiccharacters like <, >, ″, ′, or & comp_IsRepeating indicates an elementmay repeat zero or more times in the XML definition comp_levelhierarchical level of the component, starting with zero for the rootelement

As described further below, by creating an array of xmlComponentDefstructures and then setting the value of the various fields based on thedata read from the source file, XML interface 600 can create a datastructure holding all of the data necessary for the definedtransformation.

Since data is being transformed into XML format, XML interface 600 (FIG.1), in this example, is called by server 106 to help perform thetransformation. Details of XML interface 600 and its associatedcommunication handlers are described in greater detail below inconjunction with FIG. 6.

The CONNECT handler (see reference numeral 509) establishes a connectionto XML interface 600 and references an XML profile. The SEND handler(see reference numerals 510) is called before the LOAD handler (seereference numerals 512), and prepares the XML interface 600 for theload. The LOAD handler 512 loads data from a source file into an arrayelement that is passed by the example script 500. The STORE handler 514is used to create the specified target file from the target datadefinition that is passed by example script 500 to XML interface 600.The DISCONNECT handler (see reference numeral 519) disconnects from theXML interface 600.

With respect to the LOAD handler, #FILE may be used to specify the nameof the file from which the XML document may be read. #DATA may berequired to specify the array of structures that describe the XMLdocument to be read. #repeating_element_index is used on the LOAD of arepeating element and specifies the element's index in the array ofstructures.

While parsing the XML file, particular embodiments of the LOAD handlermay set the values of the various structure fields according to thefollowing guidelines: comp_name for elements and attributes, this fieldstores the name of the relevant data component comp_value initialized toNULL by the LOAD handler at the beginning of the load. May be set to thevalue contained in the document if the corresponding data component iscontained in the document comp_type elements (0), attributes (1),namespaces (2), and comments (3). comp_id, comp_parent and comp_levelcomp_namespaceURI can be specified if the document contains a namespaceURI; otherwise may be left NULL or set to a null string (“”)comp_IsRepeating set to TRUE if the element repeats; set to FALSEotherwise

With respect to the STORE handler, #FILE may be used to specify the nameof the file to which the XML document may be written (see referencenumeral 516). #DATA may be used to specify the array of structures thatdefine the components of the XML document to be created.#repeating_element_index may be used on the STORE of a repeating elementand specifies the element's index in the array of structures. Specialnotes on the use of structure fields for the STORE handler is asfollows: comp_name required for elements, attributes, and comments;namespace prefix name for namespaces comp_value optional for elements;required for attributes; ignored for namespaces and comments comp_type,comp_id, all required comp_parent and comp_level comp_namespaceURIignored for elements, attributes, and comments; required for namespacescomp_NS_Prefix optional for elements, attributes, and namespaces;ignored for comments comp_IsCDATA may be specified as TRUE if theelement value is to be wrapped in CDATA delimiters. May be set to FALSEor left NULL otherwise comp_IsRepeating set as TRUE if the elementrepeats; set to FALSE otherwise

After the generation of example script 500, mapper module 102 (FIG. 1)stores example script 500 in internal database 114 for later executionby server 106. Script manager 104 may later schedule example script 500for execution by server 106. When server 106 is ready to execute examplescript 500, it calls on XML interface 600 as shown and described belowin conjunction with FIG. 6.

Repeating Elements—Additional Information

As described above, XML Object definitions may have an additional“repeating” property added to an element in an XML Object definition(for example, Car element in XML file format 352). This property is usedto indicate if a particular element (and its children) in the XML Objectdefinition has data that repeats in the associated XML document. Therepeating property of the element is later used when the XML Objectdefinition is added to a program (see, e.g., repeating elementdefinition 408 in FIG. 4) to create “repeating element” XML definitionsfor each element with the “repeating” property. Repeating element XMLdefinitions in a program provide the means of accessing a particularrepeating element and its children for structure assignments in a scriptand processing distinct LOAD and STORE operations inside a program. Thisfeature may provide the necessary control and flexibility in the programto handle special LOAD/STORE processing required for the repeating data.

In particular embodiments of ADT 101, all elements in the XML definitionexcept for the document root and any second level elements may bedefined as repeating. The repeating property is not applicable toattributes, namespaces, and comments. The repeating property may bedesignated on child elements that are designated as repeating and soforth down the hierarchy as needed. A user may specify if an element isrepeating or non-repeating in the XML Object Definition dialog (e.g.,dialog 350 in FIG. 3B). The user may specify the repeating property froman XML Component dialog or directly from the element component in theXML Object Definition dialog via a context menu.

Moreover, in particular embodiments of ADT 101, the following rules maygovern the use of repeating elements:

1) Root and second level elements can not be designated as repeating.Consequently, second level element names are assigned unique.

2) Element names for third or lower level elements do not have to beunique as long as they are not designated as repeating.

3) Elements designated as repeating are assigned unique names for agiven parent element.

4) XML Object definitions can only have a single second level element ifthat second level element includes one or more “repeating” elements. Bycontrast, multiple second level elements may be supported for XMLobjects without repeating elements.

The main element (such as main element 406 in FIG. 4) andrepeating-element (such as the repeating Car element 408 in FIG. 4) XMLpalette objects are considered a single entity for LOAD and STOREprocessing. That is, a single XML definition structure is used in thescript to set/get values in common memory for LOAD and STORE processingof the XML document stream.

Each repeating-element definition may allow operations to be performedsuch as mappings, user constraints and process order specification andmay follow the existing rules consistent with other objects on thepalette such as tables, records, and views. When user drags and dropsthe XML object definition containing elements with the repeatingproperty onto a program palette, the main XML definition 406 and all theassociated repeating-element definitions 408 are shown, as illustratedin FIG. 4.

The name of the palette objects for repeating-element definitions may bederived from the name of the main XML definition. For example,“PersonCars” main XML Object definition is the name of the XML object inthe illustrated example. The repeating-element definition may use theelement name for the repeated-element definition followed by the parentelements up to the document root in the main XML Object definition witha period “.” separating each name.

For example, “Car.Person_Record.PersonCar_Document” is the name derivedfor the repeating-element definition for the repeating Car element shownin FIG. 4. If the final name is not unique for the palette object, anumber may be appended to the element name to make it unique (e.g.,Car2.Person_Record.PersonCar_Document). In particular embodiments, thename of the palette objects regardless of type (tables, records, views,Reusable Transformations, Lookups, etc.) will be unique in a givenprogram. The unique palette name is used to identify the object toprocessing steps (LOAD, STORE, INVOKE) in a program.

In the “Car.Person_Record.PersonCar_Document” repeating-elementdefinition 408, the fully qualified path for the Car element toPerson_Record (parent) 408 and PersonCars_Document (grand parent) isshown in the palette object name. The fully qualified path uniquelyidentifies a repeating-element definition such that the user maydifferentiate between two or more repeated-element definitions havingthe same element name but different parents. In particular embodiments,the user may be prevented from creating mappings and user constraintsto/from the parent component for Car in the parent element of therepeating-element definition in this example. That is, in suchembodiments, mappings and user constraints may not be created on the Carelement in the Person_Cars palette instance, they may be create on theCar element in the Car.Person_Record.PersonCar_Documentrepeating-element definition.

The format for repeating-element definitions is considered separate butimplicitly associated with the repeating-element definitions in therelated main XML definition and may be visible in the program palette.That is, the format maintained in the main XML definition is used todefine the corresponding repeating element definitions on the programpalette and “appear” as separate addressable palette objects (406 and408) from the user's perspective. This approach keeps all the formats ina single location in the metadata store for the repeating-elementstructure and reduces the amount of duplicated data that would be neededin an approach that uses separate repeating-element XML object metadatadefinitions in the metadata store.

When a user makes a modification to a main XML Object definition (suchas by using the XML Object editor shown in FIG. 3B), the program paletteobjects are updated to reflect the new structure. The programsynchronization routines may recreate/restructure the main/repeatingelement definitions while trying to maintain existing mappings andpreserve existing repeating element definitions if possible.Synchronization of the XML Object definitions occurs on program open,program import, and when XML objects are edited while programs are open.

The use of “repeating-element” XML Object definitions is important toallow the user to graphically see the implied DO WHILE loops that may begenerated in the script. In addition, this construct may allow the userto control how each “repeating-element” XML Object definition may beprocessed in program and resulting script. Handling of multiplerepeating elements in a single XML definition may require havingseparate LOAD/STORE loops for each repeating element. The user maycreate as many repeating-element XML definitions as necessary tocorrectly processing the XML definition.

In particular embodiments, the LOAD and STORE calls generated in thescript may have additional parameters and structures to uniquelyidentify each part of the XML definition (main or repeating-element)being processed for a particular LOAD and STORE call. This may behandled by passing the index of the “repeating element” to be processedas an additional parameter to the LOAD or STORE statements (for example,using #repeating_element_index parameter as shown at reference numeral514). A single profile may be used for the main and repeating-elementXML Object definitions on the program palette since they use the samescript structure and memory for the “related” LOAD or STORE statementsin the script. However, the “PersonCars” (main) and“Car.Person_Record.PersonCars_Document” (repeating-element) XML Objectdefinitions from above are handled using separate LOAD or STOREoperations in the script.

An XML interface 600 may save/stage the data for repeating childelements to cache both on LOAD and STORE calls in the script. Inparticular embodiments, STORE statements for repeating-elementdefinitions always stage the data to the memory. When the main record isSTOREd then all of its data, including the staged repeating elements,gets written to the XML target file. The LOAD process begins with theLOAD of the entire record. All of the non-repeating data for the rootrecord is returned, while the data for repeating child elements isstaged to a cache. On subsequent LOADs for the child portions, this datais extracted from cache and returned.

FIG. 6 is a block diagram illustrating XML interface 600 according toone embodiment of the invention. In the illustrated embodiment, XMLinterface 600 includes a plurality of communication handlers 602, aparser 604, a plurality of function calls 606, and a cache 608. Thepresent invention contemplates more, fewer, or different components thanthose shown in FIG. 6.

As described above, rather than passing data through a structure inwhich each structure field translates to a table column or file field,XML interface 600 may pass an array structure, and each element of thearray may correspond to an XML component. The array structure maycontain the information required to format an XML file, and may bederived from the XML file format 351 as described above (FIG. 3B).

For the STORE handler, XML interface 600 may simply traverse the arraystructure and write out the structure to the file in XML format. For theLOAD handler, XML interface 600 may first parse a given XML object untillocating an element corresponding to the root node of the record passedin. XML interface 600 may then parse the XML file and try to map thecomponents encountered to their corresponding fields in the array. Inone embodiment, XML components that have no corresponding field may bediscarded. Array fields that have no counterpart in the XML file may beleft NULL.

Communication handlers 602 generally provide a common function set thatenables Server 106 to interact with different databases or data formats.Communication handlers 602 may establish and terminate connections,issue queries and other suitable commands, and move data to and from adatabase or other suitable data repository. A particular interfaceconverts these common script functions into database-specific code. Inthe illustrated embodiment, communication handlers 602 include a CONNECThandler 610, a SEND handler 612, a LOAD handler 614, a STORE handler616, and a DISCONNECT handler 618.

Generally, CONNECT handler 610 allows the connection to XML interface600 in order to connect with source files and target files when server106 desires to execute a script. SEND handler 612 prepares XML interface600 for a LOAD call by passing initial information to parser 604. Thiscall may initialize the “XML file” object, and prepare the XML file tobe a source file. LOAD handler 614 works in conjunction with parser 604and may iteratively parse each successive element until the end of arecord is reached. At that point, parsing may wait and the arraydescribed above may be returned with the data portion filled in. STOREhandler 616 may cause the passed array of element structures to bewritten to the XML file format or cache 608 depending on the type ofelement being processed. The structure may contain the field names,hierarchy information, and the data. XML interface 600 may run thisstructure and generate the indicated XML to the XML file associated withthe profile. DISCONNECT handler 618 writes out any element tags that arestill pending, frees any parser resources, closes the source and targetfiles, and disconnects server 106 from XML interface 600.

Parser 604 may be any suitable computer program operable to read onedata line at a time during the LOADing of data from a source file.Parser 604 makes particular calls to function calls 606 based on thedata read in order to perform its functions. For example, function calls606 may include a comments-process function call 620, anamespace-process function call 622, a CDATA-process function call 624,a start element function call 626, and an end element function call 628.

Cache 608 may be any suitable storage unit or database and may take theform of volatile or non-volatile memory including, without limitation,magnetic media, optical media, random access memory, read-only memory,removable memory, or any other suitable local or memory component. Inthe illustrated embodiment, cache 608 includes a stack 630, a componentlist 632, and a repeating component list 634. Stack 630 functions tokeep track of where parser 604 is in an XML tree structure when XMLinterface 600. Component list 632 temporarily caches data from aparticular source file or target file during a LOAD call and repeatingcomponent list 634 temporarily caches data for the repeating elementsduring a particular LOAD call. This is described in further detail belowin conjunction with FIGS. 7A and 8 below.

To support STORing of repeating elements, repeating records, and nestedrepeating records, example script 500 stores the repeating data firstfollowed by the non-repeating data. First, example script 500 determineswhat type of data is being processed, e.g., a repeating record,repeating element or non-repeating data. Example script 500 determinesthis by using a repeating index passed on the STORE call from examplescript 500. Next, if example script 500 determines that a repeatingrecord is being processed then the size of the record (that is, thestart and end indexes of the record) is calculated.

Repeating component list 634 maintains the list of all active repeatingrecords being processed. Any time a repeating record is parsed, theparsing routines add a new reference to repeating component list 634.When a repeating component is parsed, repeating component list 634 isscanned to determine if the record is a member of this list (has beenprocessed before). If so, XML interface 600 identifies the parent of therepeating component from repeating component list 634. On STORE of themain component, the non-repeating data is recorded in component list632. A call is subsequently made to STORE handler 616 to write out theXML record.

While processing a repeating record, a check for nested repeating recordscans for any nestings of repeating data. All repeating data may havetheir own LOADs, which may each cause a new in-memory cache data area tobe added in the component. Parsing routines, while building the list,will save the component data in the in-memory cache and LOAD handler 614will then process the data if the Component Ids match a Component Idthat is to be processed next.

FIGS. 7A-7B and 8 illustrate example operation of XML interface 600 inexecuting a transformation script such as the one shown in FIGS. 5A-5C.In particular, FIG. 7A illustrates example operation of XML interface600 in executing an example script to transform input data stored inrows from a database table into an XML output file, while FIG. 8illustrates example operation of XML interface 600 in executing anotherexample script to transform input data stored in an XML-format sourcefile to rows in a database table. In general, however, XML interface 600may be configured to transform input data from an XML-format source fileinto a target file of any appropriate format. As indicated above otherpossible target file-to-source file combinations may include, but arenot limited to XML file-to-XML file, flat file-to-XML file, and XMLfile-to-flat-file transformations.

FIG. 7A is a flowchart illustrating steps that may be taken by server106 and XML Interface 600 in executing the example script 500 shown inFIGS. 5A-5C to transform a portion of a particular set of databasetables (referred to generically here as the “source files” for thistransformation) into an XML document having a particular format(referred to generically as the “target file” in this transformation).More specifically, FIG. 7A illustrates the transformation of a pluralityof PERSON rows from a PERSON table and a plurality of associated CARrows from a CAR table into a plurality of XML records that each includesinformation about a particular person and one or more cars associatedwith that person.

The process begins at step 710 with server 106 initiating execution ofthe generated script. At step 720, server 106 creates array 502describing the target data definition (here, the PersonCars XML Objectdefinition) and initializes one or more values of its various arrayelements. For example, using example script 500 illustrated in FIGS.5A-5C as an example, during initialization server 106 creates the“PersonCars_Document” array and sets the “name,” “type,” “id,” “parent,”and “level” of each element of the “PersonCars_Document” array based onthe PersonCars XML data definition that was supplied at the time ofscript creation.

At step 730, the relevant interfaces CONNECT to the source file andtarget file. In particular here, XML interface 600 issues a CONNECT toconnect to the XML target file, while database interface 116 issuesCONNECTs to the “Person” table and the “Cars” table in the relationaldatabase associated with database interface 116. At this point, server106 may also initialize other operational variables, clear temporarymemory, and/or perform any other steps appropriate to facilitate inputand output to the relevant files.

The appropriate interface then begins reading data from the source file.Here, database interface 116 begins reading data from the relevantdatabase tables. In particular, database interface 116 transmits a SENDrequest to the PERSON table to request all PERSON rows at step 740.These PERSON rows may then be buffered in temporary memory by databaseinterface 116 until needed.

At step 750, server 106 determines whether another PERSON row can beloaded from those stored in memory by issuing a LOAD call on the PERSONtable. If server 106 determines that no more rows remain to beprocessed, server 106 continues operation at step 800. If, instead,server 106 determines that additional rows remain to be processed,server 106 accesses the next remaining PERSON row and transforms thedata in this PERSON row for output to the target XML file. As part ofthis process, server 106 may also iteratively process any repeating dataelements associated with this PERSON row.

For example, in the illustrated example, server 106 transmits a SENDrequest to the CARS DBMS table to request the CAR rows having aparticular ID value at step 760. These CAR rows may then be buffered intemporary memory. At step 770, server 106 determines whether any CARrepeating elements remain to be processed by issuing a LOAD on theidentified CAR rows. If server 106 determines that no more CAR rowsremain to be processed for this particular PERSON row, server 106continues operation at step 790. If, instead server 106 determines thatadditional CAR rows remain to be processed for this particular PERSONrow, server 106 accesses the data in the next remaining CAR row andissues a STORE for the relevant CAR row at step 780. As a result of theSTORE, data from this CAR row will be cached internally by XML interface600 along with other data previously cached for the repeating XML datacomponent type associated with these CAR rows. In particularembodiments, this data may saved in the buffer until being written tothe target XML file when the STORE for the parent data component isprocessed (e.g., at step 790 in this example). Additionally, inparticular embodiments, server 106 may also format or otherwise modifythe data extracted from the relevant CAR rows to match the target datadefinition associated with the target file. For example, server 106 maymodify the format of a model year stored in a particular CAR row tomatch the target year format associated with the data definition for thetarget XML file.

Once the appropriate data for a particular data component in the targetXML file has been buffered, including any appropriate repeatingelements, XML interface 600 writes the data component to the target XMLfile. More specifically, in the illustrated example, XML interface 600writes out a Person_Car XML record to the target XML file at step 790.Operation then returns to step 750 and server 106 attempts to LOADanother PERSON row.

Once data for all of the XML data components has been LOADed from theappropriate source files, database interface 116, XML interface 600 andserver 106 may complete any steps appropriate to finalize thetransformation and close the source file and target file. In theillustrated example, as part of this process, server 106 performs awrite of the last remaining XML data component to the target XML file,and XML interface 600 and database interface 116 issue DISCONNECTs tothe target XML file and source database tables respectively at step 800.At step 810, server 106 completes execution of the example script andterminates operation with respect to this particular datatransformation. The target XML file may now be viewed by an appropriateXML editor.

FIG. 7B is an example output of the example method of FIG. 7A accordingto one embodiment of the invention. As indicated in FIG. 7B, an XML file798 is illustrated in which car data for particular persons are arrangedin an XML format. This data was extracted from database tables, such assource tables 404 a, 404 b (FIG. 4).

FIG. 8, as noted above, illustrates an example operation of XMLinterface 600 in executing another transformation. In particular, FIG. 8illustrates operation of server 106 and XML Interface 600 in executinganother example script (not shown) generated to transform the data inthe XML file shown in FIG. 7B (representing the source file for thistransformation) into an arrangement of DBMS rows (representing thetarget files in this transformation) having a particular format.Moreover, FIG. 8 provides additional detail for some of the high-levelsteps identified in FIG. 7, with respect to how these steps might beimplemented in a particular embodiment of system 100.

The process begins at step 900 with server 106 initiating execution ofthe script generated to perform this transformation. At step 910, server106 creates an array describing desired data that will be stored in thetarget files (here, the fields of the PERSONS rows and the associatedCARS rows) based on the target data definition defined by the executingscript. Server 106 additionally initializes one or more values of thevarious array elements in this array.

At step 920, the relevant interfaces CONNECT to the source file andtarget file. In particular here, XML interface 600 issues a CONNECT toconnect to the XML source file, while database interface 116 issuesCONNECTs to the “Person” table and the “Cars” table in the relationaldatabase associated with database interface 116. At this point, server106 may also initialize other operational variables, clear temporarymemory, and/or perform any other steps appropriate to facilitate inputand output to the relevant files.

At step 930, XML interface 600 begins parsing the XML source file 798which includes multiple XML data records. While parsing, XML interface600 determines at step 940 whether XML interface 600 has reached thestart of an XML data component (e.g., based on the detection of a startdelimiter in the parsed data). When XML interface 600 determines that ithas detected the beginning of an XML data component, XML interface 600determines, at step 950, whether this data component should be includedin the target file. In particular embodiments, XML interface 600 maydetermine whether to include the detected data component by traversingthe array and determining whether the “Name” field of any array elementmatches the name of the detected XML component. If not, XML interface600 discards the detected data component at step 960 and proceeds withparsing at step 1060.

If, instead, XML interface 600 is able to match the name of the detectedXML component to the “Name” field of one of the array elements, XMLinterface 600 processes the data component for inclusion in the array.As part of this process, XML interface 600 determines, at step 970,whether the detected data component has any children components. If so,XML interface 600 parses and processes the children components, at step980, in a similar fashion deciding whether each should be included inthe array. (Although shown, for the sake of simplicity, as a single box,this process may, depending on the hierarchy of the detected children,be an iterative process that follows a flow similar to that taken toprocess the detected parent component.)

XML interface 600 then determines at step 990 whether the detected datacomponent is of a repeating component type. In particular embodiments,XML interface 600 may determine this by checking a field, such as an“IsRepeating” field of the array 502 illustrated in FIG. 5A. If XMLinterface 600 determines that the detected data component is not of arepeating component type, XML interface 600 sets a field (e.g., a“Value” field) of the matched array element to the value of the detecteddata component at step 1000. As a result, data from matched datacomponents will be stored in their corresponding array elements.

If XML interface 600 determines that the detected component is of arepeating component type, XML interface 600 stores the value of thedetected component in a temporary buffer at step 1010. XML interface 600then determines if there are additional data components of the samecomponent type immediately following the detected component at step1020. If so, XML interface 600 stores the next repeating data componentin the buffer at step 1030 and adds a pointer back to the previousrepeating element at step 1040. XML interface 600 then returns to step1020. Once all repeating elements of that component type have beenstored in the buffer, XML interface 600 adds a pointer to the lastrepeating element of that type to a field (e.g., a “Value” field) of thematched array element (i.e., the array element that originally matchedthe name of the first data component of this repeating type) at step1050. XML interface 600 then returns to parsing the remainder of thesource XML file.

At step 1060, the interface 108 associated with the target file, in thiscase data base interface 116, issues a STORE on any non-repeating datafrom the currently-parsed data component, thereby writing thenon-repeating data to the target file. XML interface 600 then determineswhether the currently-parsed data component has any repeating childrenat step 1070. If so, XML interface 600 retrieves, from the memorybuffer, data from one of the repeating children and database interface116 issues a STORE on this repeating child data, writing the data to thetarget file at step 1080. XML interface 600 and database interface 116repeat this process until all of the repeating children have beenstored, returning to step 1070 until no more children remain in memory.

As one example, in the described configuration, the target filesrepresent associated rows in the PERSONS and CARS tables. As a result,database interface 116 may write the values stored in a particular field(e.g., a “Value” field) of the array elements associated withnon-repeating component types to a PERSON row in the PERSON table.Database interface 116 then accesses the memory location identified bythe pointer stored in the array elements associated with any repeatingcomponent types, here the CARS data components, and writes the valuesstored at that location to the target files in the appropriate mannerbased on the type of target files. In the illustrated example, databaseinterface 116 creates a CAR row for each CAR data component and adds itto the CARS table. Because the associated PERSON row was already createdand added to the PERSONS table, database interface 116 can alsoincorporate an ID identifying the associated PERSON row into each of thenewly-created CAR rows.

After database interface 116 stores or saves the appropriate data, XMLinterface 600 determines whether it has completed parsing the XML sourcefile at step 1090. For example, in particular embodiments, XML interface600 may determine if it has completed parsing the last data componentbased on whether or not XML interface 600 has detected an end delimiterassociated with the root data component. If XML interface 600 has notcompleted parsing the XML source file, XML interface 600 returns to step930 and continues parsing the XML source file.

If, instead, XML interface 600 determines that it has finished parsingthe XML source file, XML interface 600, database interface 116, andserver 106 may complete any steps appropriate to finalize thetransformation and close the source file and target file. As part ofthis process, XML interface 600 and database interface 116 issueDISCONNECTs to the target XML file and source database tablesrespectively at step 1100. At step 1110, server 106 completes executionof the example script and terminates operation with respect to thisparticular data transformation.

FIGS. 9-12 illustrate the operation of particular embodiments of datatransformation systems that may utilize web services in various ways tosupplement the transformation functionality described above with respectFIGS. 1-8. FIG. 9 illustrates the operation of a system 120 capable ofgenerating data transformation scripts that are similar to thosedescribed above but that may be executed as web services. Meanwhile,FIGS. 10-12 illustrate the operation of a system 160 capable ofgenerating data transformation scripts that are also similar to thosedescribed above but that may invoke web services offered by otherservers to complete requested data transformations. The incorporation ofweb service features into ADT systems such as these may increase theflexibility of these ADT systems and may further reduce the amount ofdesign required of ADT users.

FIG. 9 illustrates a system 120 that allows transformation scripts(referred to herein as “web-invoked scripts 125”) to be executed via aweb service call. In general, the ability to execute such web-invokedscripts 125 as a web service may allow the existing program executionarchitecture of system 100 to be accessed by remote devices through acall handled by a web server 124 with execution-specific data passed inand out. Additionally such a configuration may allow system 120 toschedule the execution of a web-invoked script 125 for a future timeand/or add other forms of flexibility to the transformationfunctionality described above. As shown in FIG. 9, system 120 includesADT 121, web server 124, client 122, and network 123. Moreover, webserver 124 includes network interface module 128 and messaging module129, while ADT 121 includes mapper module 142, script manager 144,server 146, interfaces 148, and internal database 154. Except asexplicitly noted below, mapper module 142, script manager 144, server146, interfaces 148, and internal database 154 may, in particularembodiments, all operate in a similar fashion to that described abovewith respect to similarly-labeled components of ADT 101. In addition,ADT 121 includes scheduler 138.

In general, web server 124 receives service requests 130 from one ormore clients 122 requesting data transformation service. Web server 124,in turn, requests these data transformation services from server 146.After server 146 completes the desired data transformation usingweb-invoked scripts 125, web server 124 may also transmit a serviceresponse 131 to client 122 with the results of the requested datatransformation services. Web server 124 may represent any appropriatecombination of software and/or hardware suitable to provide thedescribed functionality. For example, in particular embodiments, webserver 124 may represent a server running Apache Tomcat. Additionally,in particular embodiments, web server 124 may represent the samephysical component as ADT 121. For example, ADT 121 and web server 124may represent applications running on the same computer. Alternatively,as suggested by dotted line 127 between ADT 121 and web server 124, inparticular embodiments, ADT 121 and web server 124 may representseparate physical devices, or applications running on separate physicaldevices, operable to communicate with one another.

Client 122 may represent a software application executing on a suitablyconfigured personal computer (PC), networked terminal, or any otherappropriate device capable of accessing web services. Although thedescription below focuses on embodiment of system 120 in which client122 is running on a separate physical device remote from ADT 121, inparticular embodiments, client 122 may represent a software applicationrunning on the same computer as ADT 121 and/or web server 124. Network123 may represent a local area network (LAN), portions of the Internet,or any other suitable public or private communications network. Webserver 124, server 146, network interface module 128, and messagingmodule 129 represent any appropriate combination of hardware and/orsoftware suitable to provide the described functionality.

In operation, ADT 121 generates web-invoked scripts 125 to performuser-defined data transformations. A user of system 120 may define therelevant transformations and ADT 121 may generate the correspondingweb-invoked scripts 125 in any appropriate manner. In particularembodiments, a GUI similar to the one described in FIGS. 3A-3C and4A-4B, may be modified for used in creating web-invoked scripts 125. Forexample, the GUI of FIG. 4 may be modified to include a “Web Services”select box that the user may select, when creating a script using theprocess described above with respect to FIG. 4, to indicate that thetransformation currently being defined is intended to be provided as aweb service. When the user selects the box to indicate that this scriptis to be executed as a web service, the GUI may enable a number of GUIinputs for defining web-invoked scripts 125. As a result, in particularembodiments, server 146 may receive information identifyingweb-service-capable transformation web-invoked scripts 125 in a similarfashion to that described above with respect to scripts 115.

Furthermore, web-invoked scripts 125 may also be similar in structure toscripts 115 described above with respect to FIGS. 5A-5C with suitablemodifications made to the code to allow web-invoked scripts 125 toutilize data from web service messages as the target and source files ofthe relevant data transformation. For example, in particularembodiments, the LOAD and STORE functions for the XML interfacedescribed above accept a “#ws_xml” field that indicates whether thetarget or source file, respectively, should be retrieved from a webservice request or transmitted as a web service response. Forweb-invoked scripts 125 this field is set true when the web-invokedscript 125 is generated.

ADT 121 may be configured to generate web-invoked scripts 125 thataccept input data from client 122 and/or provide output data to client122. In particular embodiments, a particular web-invoked script 125 maybe configured not to accept input data from client 122 and/or notprovide output data to client 122, depending on the configuration of ADT121 and the transformation defined for the relevant web-invoked script125. In such embodiments, web-invoked scripts 125 may be configured toretrieve input data from local memory and/or to store output data tolocal memory as an alternative to exchanging data with client 122.

Additionally, once a particular web-invoked script 125 has beengenerated, system 120 may be configured to create a service definition134 that can be used to provide client 122 with information regardingthe appropriate manner for communicating with web server 124 to requestthe associated transformation as a web service. Service definition 134may represent a Web Services Description Language (WSDL) file, and XMLschema, and/or any other appropriate collection of informationidentifying for client 122 the appropriate inputs to use and/or outputsto anticipate when requesting the execution of the relevant web-invokedscript 125 as a web service. In particular embodiments, servicedefinition 134 may be generated by mapper module 142 when a particulardata transformation is defined and a user requests the associated scriptbe made available as a web service. In particular embodiments, when theuser chooses to have service definition 134 generated, mapper module 142may generate this service definition 134 “automatically” in the sensethat mapper module 142 may generate service definition 134 based onmappings in the program palette without any additional input from theuser beyond the initial request for generation of service definition134. Server 146, web server 124, or any other appropriate component ofsystem 120 may then publish service definition 134 at an appropriateregistry (e.g., in a Universal Description, Discovery, and Integration(UDDI) repository) for access by clients 122 in system 120, transmitservice definition 134 to requesting clients 122, or otherwise makeservice definition 134 available to clients 122.

After one or more web-invoked scripts 125 have been generated, client122 may request data transformation services by transmitting a servicerequest 130 to web server 124 over network 123. Service request 130 mayrepresent a Simple Object Access Protocol (“SOAP”) message, anElectronic Business using eXtensible Markup Language (“ebXML”) message,or a message of any other type or format appropriate for requesting webservices. Service request 130 may include request data 132, such as oneor more XML data components; one or more DBMS table entries, rows, orcolumns; or any other appropriate collection of data. In particularembodiments, web server 124 may also be configured to support scheduledprocessing of service requests 130, and service request 130 may alsospecify that the request is for a scheduled execution and supply arequested execution time 139.

A network interface module 128 of web server 124 receives servicerequest 130 and processes service request 130 to facilitate execution ofthe request by ADT 121. Network interface module 128 may represent anyappropriate hardware and/or software suitable to provide the describedfunctionality. In particular embodiments, network interface module 128comprises a Java servlet that handles initial message decoding and aJava Native Interface (JNI) for communicating with messaging module 129.

Network interface module 128 performs any appropriate decoding ofservice request 130 and extracts request data 132. For example, as notedabove, in particular embodiments, service request 130 represents a SOAPmessage, and network interface module 128 performs initial SOAP decodingof service request 130 and extracts request data 132 from the servicerequest 130. Network interface module 128 then passes request data 132and any other appropriate information from the service request 130, suchas the requested execution time 139, to a messaging module 129 of webserver 124. In particular embodiments, a Java servlet of networkinterface module 128 passes the request data 132 to messaging module 129using JNI.

Messaging module 129 receives request data 132 from network interfacemodule 128 and interacts with server 146 to facilitate completion of therequested data transformation. In particular embodiments, messagingmodule 129 may interact with server 146 using threaded messages 135. Forexample, in particular embodiments, web server 124 and server 146 may beoperating in a system using Computer Associate's Platinum EnterpriseCommunicator (PEC), and threaded messages 135 may represent PEC messagestransmitted between messaging module 129 and server 146. Additionally,threaded messages 135 may each include a thread identifier and/or asource identifier to allow both web server 124 and server 146 tocoordinate communication and processing related to service request 130.

Thus, after receiving the decoded service request 130, messaging module129 transmits one or more threaded messages 135 to server 146 to requesttransformation services. As part of one or more of the threaded messages135, messaging module 129 communicates the request data 132 to server146. In particular embodiments, server 146 may be able to identify,based on the name and/or structure of request data 132, an appropriateweb-invoked script 125 to execute from among a plurality of web-invokedscripts 125 currently stored in internal database 154. Alternatively,messaging module 129 may also communicate additional information toserver 146 to allow server 146 to determine the appropriate datatransformation to be completed. For example, in particular embodiments,messaging module 129 may also include the name of a particularweb-invoked script 125 to be executed by server 146 in completing thedesired data transformation.

Server 146 receives threaded messages 135 from web server 124 andinitiates one of web-invoked scripts 125 using request data 132 as thesource file. As noted above, in particular embodiments, web-invokedscripts 125 may be generated with LOAD statements that are configured toreceive input data from service requests 130 and/or with STOREstatements that are configured to write output data to service responses131. Once server 146 has completed execution of the appropriateweb-invoked script 125, server 146 may communicate back any appropriateoutput, including any response data 133 generated as a result of theexecution of the relevant web-invoked script 125, to the messagingmodule 129 of web server 124 in one or more threaded messages 135.Alternatively, server 146 may store any output of the executedweb-invoked script 125 locally and may not communicate any response data133 back to web server 124. In particular embodiments, response data 133for all scheduled data transformations may be stored local to ADT 121,and clients 122 may not receive any response data 133 when requestingscheduled data transformation services.

Messaging module 129 receives threaded messages 135 from server 146 andforwards threaded message 135 (or information obtained from the threadedmessage 135) to network interface module 128. Network interface module128 then determines an appropriate client 122 to which a serviceresponse 131 should be transmitted. Network interface module 128generates a service response 131 and transmits service response 131 tothe client 122 that originally transmitted the corresponding servicerequest 130. Service response 131 includes the response data 133received from server 146. As a result of this process, a remote userusing client 122 may be able to remotely access and utilize the datatransformation capabilities of ADT servers using web service calls.

Additionally, as suggested above, particular embodiments of web server124 and server 146 may support scheduled execution of the datatransformations requested via web service calls. More specifically, webserver 124 may receive service requests 130 that include an executiontime 139. Server 146 may then execute the web-invoked script 125corresponding to the requested transformation at a time determined basedon execution time 139. For example, particular embodiments of server 146may include a scheduler 138. Messaging module 129 may determine, basedon the inclusion of execution time 139 in particular service requests130, that those requests are to be scheduled for execution at a latertime. For these service requests 130, messaging module 129 may transmit,to scheduler 138, threaded messages 135 that include the request data132 and execution time 139. Scheduler 138 may then store request data132, execution time 139, and any other appropriate information to allowserver 146 to properly execute the desired transformation. At anappropriate time, scheduler 138 may then initiate the desiredweb-invoked script 125 or instruct other components of server 146 toinitiate the desired web-invoked script 125. If the service request 130included a request data 132, scheduler 138 may provide this to therelevant components as well.

Additionally, scheduler 138 may determine the appropriate time to beginexecution of the relevant script based in any suitable manner onexecution time 139. For example, execution time 139 may represent aninitiation time at which scheduler 138 will execute the web-invokedscript 125, a completion time at which the web-invoked script 125 mustbe completed, or a priority level that server 146 will use to order thevarious tasks server 146 currently has scheduled.

Thus, system 120 provides a flexible solution for data transformationsolutions. As a result, system 120 may be capable of providing a varietyof services to remote users using standardized interfaces. Moreover,system 120 may be configured to provide clients 122 information as tothe services offered by server 146 and the proper manner for accessingthose services through the use of a service definition 134.Additionally, system 120 may be capable of timing the execution of therelevant data transformation based on the desires of the requestinguser. As a result, system 120 may provide a number of operationalbenefits.

FIG. 10 illustrates a system 160 that utilizes web services to providefunctionality for transformation programs (referred to herein as“web-utilizing scripts 155”). In general, system 160 allows a user toidentify a web service to be called to provide all or a portion of aparticular data transformation. In particular embodiments, web-utilizingscripts 155 may be identical to particular types of scripts 115described above with respect to FIGS. 1-8 with the added ability toinvoke external web services based on a defined data transformation. Theweb service is identified during mapping and a particular web-utilizingscript 155 is generated to perform the defined data transformation usingthe specified web service.

In the illustrated embodiment, system 160 includes an ADT 181, a webserver 162, a network 163, and a proxy server 164. ADT 181 includesserver mapper module 182, script manager 184, server 186, interfaces188, and internal database 194. Except as explicitly noted below, mappermodule 182, script manager 184, server 186, interfaces 188, and internaldatabase 194 may, in particular embodiments, all operate in a similarfashion to that described above with respect to similarly-labeledcomponents of ADT 101. In addition, ADT 181 includes service interfacemodule 183.

Service interface module 183 is responsible for receiving inputparameters from server 186 during execution of a web-utilizing script155, packaging input parameters, invoking the web service, andun-packaging output parameters. Although the description below focuseson examples in which ADT 181 transmits input data to the requested webservice and receives particular output data back from the web service,particular web services may be configured to receive no input dataand/or to transmit no output data back to ADT 181. In particularembodiments, service interface module 183 includes a dynamically-linkedC++ library capable of receiving appropriate input data (as describedfurther below), structuring the input data in an appropriate manner forthe particular web service to be invoked, and communicating the data toa SOAP tool, which in turn transmits a SOAP message containing the datato the designated web server 162. In particular, the dynamically-linkedlibrary may support a generic web execution function capable of callingany web service identified by function parameters 174 received by thegeneric web execution function. In general, however service interfacemodule 183 may represent any suitable hardware and/or softwareappropriate to communicate with web servers and utilize web services.

Web server 162 may represent any appropriate component providing anyfunctionality (generically referred to herein as “web services 175”)that can be utilized through the transmission of a suitably-structuredrequest and the receipt of a corresponding response. For example, inparticular embodiments, web server 162 may comprise an Apache Axis orMicrosoft NET server supporting web services 175. In general, however,web server 162 may represent any appropriate combination of softwareand/or hardware suitable to provide the described functionality.

In operation, mapper module 182 receives information defining a datatransformation that utilizes a web service 175 and generates aweb-utilizing script 155 that invokes the relevant web service 175 toexecute the defined transformation. Web-utilizing scripts 165 may besimilar in content and operation to scripts 115 described above withrespect to FIGS. 1-8, but with the addition of web service calls to, inpart, transform data extracted from source data files into data to bewritten into target data files. An example of a particular web-utilizingscript 165 is shown in FIG. 12 and discussed below.

More specifically, mapper module 182 allows a user to design atransformation program to transform data via mappings from one or moresource files to one or more target files utilizing one or more specifiedweb services. In particular embodiments, the mapping may be defined viaa GUI having a program palette (as shown in FIG. 11) in which a user isallowed to select a source object definition, a target objectdefinition, and a web service definition 178. In particular embodiments,web service definition 178 may comprise information defining inputsexpected by and outputs transmitted by the corresponding web service175. The user may then drag and drop data components from a graphicalrepresentation of the source object definition into a graphicalrepresentation of the web service definition 178, and then drag and dropoutputs of web service definition 178 into a graphical representation ofthe target object definition in order to define the desiredtransformation.

As part of the information the user provides the define thetransformation, the user may identify a source object definition, atarget object definition, and a web service definition 178 that will beutilized as part of the transformation. With respect to the web service,the user may identify such information as the web service UniformResource Indicator (URI) location, the method to invoke for the webservice, a proxy server to contact, and/or any other appropriateinformation to facilitate communication between the web-utilizing script155 and the relevant web service 175.

The user may also provide a web service definition 178 to be displayedin a GUI associated with ADT 181. Web service definition 178 defines theinputs and outputs associated with a particular web service 175 and anyadditional information service interface module 183 may need to requestweb services from a particular web server 162. The user may provide webservice definition 178 to ADT 181 in any suitable manner. In particularembodiments, ADT 181 may be capable of parsing a WDSL, XML schema, orother suitable description of a web service to automatically generateweb service definition 178. In alternative embodiments, the user mayprovide web service definition 178 to ADT 181 manually, identify apreviously-saved web service definition 178, or provide web servicedefinition 178 to ADT 181 in any appropriate manner.

After the user has provided one or more source object definitions, oneor more target object definitions, and one or more web servicedefinitions 178, the user specifies mappings between source filecomponents and target file components, between source file componentsand web service inputs, and/or between web service outputs and targetfile components. Once the user has provided the appropriate informationto define the transformation, mapper module 182 generates aweb-utilizing script 155 to perform the defined data transformation.This web-utilizing script 155 will include calls to the identified webservice 175 through service interface module 183.

Web-utilizing scripts 155 may be stored by mapper module 182 until auser requests the corresponding data transformation be executed. When aparticular web-utilizing script 155 is executed, server 186 will executethat web-utilizing script 155 and as part of this process will performdata transformations that utilize web services. More specifically, inparticular embodiments, web-utilizing script 155 will include a call toservice interface module 183 that will transfer various functionparameters 174 to service interface module 183 for use in generating aservice request 166. The exact data that is included in functionparameters 174 may depend on the configuration of server 186, web server162, and/or service interface module 183. In particular embodiments,function parameters 174 includes the URI for the relevant web service,the method to be invoked by that web service, any input data 168, and/orempty output data structures to be filled with the results of the webservice. Input data 168 may represent data of any appropriate formatincluding, but not limited to, one or more XML data components, one ormore database rows, or any portion of a flat file. Similarly, the outputdata structures may represent empty XML data components, empty rows,empty variables, or any other suitable structure for holding data.Particular data transformations may not require input data or returnoutput data, and the web service calls executed by web-utilizing scripts155 may be tailored to reflect this fact.

After receiving function parameters 174 from server 186, serviceinterface module 183 invokes the designated web service 175. Inparticular embodiments, this process includes connecting to the URIspecified in the function parameters 174, transferring the input data168 into a service request 166, and transmitting service request 166 toa particular web server 162 associated with the specified URI. Inparticular embodiments, service request 166 represents a SOAP messagethat contains input data 168. Alternative embodiments may use otherappropriate communication protocols to request web services andotherwise interact with web server 162.

Web server 162 receives service request 166 and unpacks data included inservice request 166. Web server 162 identifies the appropriate method toinvoke based on information in the received service request 166 andinvokes the identified method with respect to input data 168, if any,that is included in service request 166. After the method has beeninvoked, web server 162 transmits output data 169, if any, back toservice interface module 183 as part of a service response 167.

Service interface module 183 receives service response 167 and completesthe data transformation requested by the user. In particular, serviceinterface module 183 receives service response 167, performs anyappropriate unpacking of service response 167, and transfers output data169, if any, into the empty data structures passed to service interfacemodule 183. Service interface module 183 then transmits the output data169 back to server 186 for use in completing execution of web-utilizingscript 155. In particular embodiments, service interface module 183transmits the output data 169 to server 186 as the return value of thegeneric web execution function originally called by service interfacemodule 183.

In addition, particular embodiments of service interface module 183 maysupport the use of proxy servers 164 (e.g., for use with firewalls) tocommunicate with web server 162. More specifically, in particularembodiments of system 160, function parameters 174 passed to serviceinterface module 183 may include information that allows serviceinterface module 183 to identify a proxy server 164 to use when invokinga particular web service. This information may include a proxy serverhost name, an appropriate port number to use when communicating with thedesignated proxy server 164, a user identifier recognized by thedesignate proxy server 164, a password associated with the designatedproxy server 164, and/or any other suitable information to facilitatecommunication between server 186 and proxy server 164. Alternatively,some or all of this type of information may be stored in a proxyprofile, and function parameters 174 may identify the appropriateprofile to use when invoking the relevant web service. Server 186 maythen direct any communication with web server 162 through proxy server164. This may allow users to utilize publicly-available web services 175for data transformations without compromising the security of their ownsystems.

Thus, system 160 provides additional flexibility and user ease withrespect to the execution of data transformations. By allowing a user toaccess and utilize available web services, system 160 may further reducethe amount of programming and/or design the user must complete.Automated parsing of web service definitions may improve ease-of-useeven further in some embodiments. Additionally, particular embodimentsmay be configured to interact with web service through a proxy server,thereby making web-service functionality available for datatransformations without compromising security.

FIG. 11 illustrates an example screen shot 1200 illustratingfunctionality of a GUI that may be used, in particular embodiments ofsystem 160, to define a data transformation that utilizes one or moreweb services. In particular, FIG. 11 illustrates a program palette 1202that allows a user to design a desired data transformation from one ormore source files to one or more target files using one or more webservices. The illustrated embodiment includes a graphical representationof a source object definition 1204, a target object definition 1206, areusable transformation 1207, and a web service definition 1208.Although FIG. 11 and the description below focus, for purposes ofillustration, on a particular type of GUI that allows a user to entercertain information in a specific manner, system 160 may utilize anyappropriate form of GUI to facilitate interaction with a user. Moreover,particular embodiments of system 160 may include no GUI and users mayenter information manually, instruct system 160 to retrieve savedinformation, and/or provide information to system 160 in any othersuitable manner. Furthermore, FIG. 11 illustrates, for purposes ofexample, a scenario in which a web service is utilized to transformportions of a source database table into a target database table.Nonetheless, the described techniques may be utilized with target andsource files of any appropriate format including XML files and flatfiles.

With respect to the particular GUI illustrated in FIG. 11, a user maybegin entering information by dragging and dropping of objectdefinitions 1204 and 1206, reusable data transformation 1207, and webservice definition 1208 into program palette 1202. For example, in theillustrated example, a user has dragged a source object definition 1204named “StockCompanies” and a target object definition 1206 named“StockQuotes” into program palette 1202. Additionally, the user hasdragged a web service definition 1208 named “GetQuoteWS” and a reusabledata transformation 1207 named “GetCurrentTime” into program palette1202. The “GetQuoteWS” web service definition 1208 in the illustratedexample is associated with a web service 175 that provides stock pricesfor specified stock symbols. In addition, particular embodiments of ADT181 may support scripted, reusable functionality, such as one or morereusable data transformations 1207, that provide data transformationsand/or custom outputs that may be used to execute frequently-used datatransformations or to generate frequently-used data outputs. Herereusable data transformation 1207 provides the current time as output1218 b.

Once the appropriate object definitions 1204 and 1206, any reusabletransformations 1207, and any web service definitions 1208 have beenintroduced to program palette 1202, the user may define connections 1210between specific data components 1212 of source object definitions 1204and target object definitions 1206 and particular inputs 1216 andoutputs 1218 of web service 175. In particular embodiments, the user maydo this by dragging and dropping components of source object definition1204 onto inputs 1216 of web service definition 1208 or reusable datatransformation 1207 or onto components of target object definition 1206,and by dragging and dropping outputs 1218 of web service definition 1208or reusable data transformation 1207 onto components of target objectdefinition 1206.

For example, in the illustrated scenario, the user has dragged the“Stock Symbol” data component 1212 b from the “StockCompanies” sourceobject definition 1204 to the “symbol” input 1216 of “GetQuoteWS” webservice definition 1208 forming connection 1210 a. The user has alsodragged the “Stock Symbol” data component 1212 b from the“StockCompanies” source object definition 1204 to the “StockSymbol” datacomponent 1212 c of the “StockQuotes” target object definition 1206forming connection 1210 b. Additionally, the user has dragged the“Result” output 1218 a of the “GetQuoteWS” web service definition 1208to the “StockPrice” data component 1212 e of the “StockQuotes” targetobject definition 1206 forming connection 1210 c. The user also hasdragged the “timestamp” output 1218 b of the “GetCurrentTime” reusabletransformation 1207 to the “QuoteDateTime” data component 1212 d of the“StockQuotes” target object definition 1206 to create connection 1210 d.

After moving the relevant object definitions 1204 and 1206, web servicedefinitions 1208, and reusable data transformations 1207 to programpalette 1202 and creating the appropriate connections 1210, the user maythen utilize system 160 to generate a web-utilizing script 155 toimplement the defined transformation. In particular embodiments, the GUIfor defining the transformation may additionally include a “GenerateScript” button or other input the user can use to request aweb-utilizing script 155 be generated from the mappings the user hascreated on program palette 1202. Alternative embodiments may use anyappropriate mechanism for the user to request script generation.

FIGS. 12A and 12B collectively illustrate an example of a web-utilizingscript 155 that may be generated by a particular embodiment of system160 based on the transformation defined in FIG. 11. Although FIGS. 12Aand 12B illustrate one example of a particular web-utilizing script 155that may be generated to implement the transformation defined in FIG.11, alternative embodiments of system 160 may generate other types ofweb-utilizing scripts 155 to implement the illustrated transformationbased on the configuration and capabilities of system 160. For purposesof description, the web-utilizing script 155 is broken into multiplesections 1310.

Section 1310 a identifies a code fragment that defines various variablesand structures to be used during execution of web-utilizing script 155.Section 1310 b generates the array that will be used to pass XML datafrom server 186 to service interface module 183 as part of functionparameters 174. This array will store any input data that will betransmitted by service interface module 183 to the appropriate webservice 175. ser. Section 1310 c constructs the data structure that willbe used to pass information about the relevant web service to serviceinterface module 183 as another part of function parameters 174. Section1310 d constructs the empty data structure that will be filled withoutput data 169 and passed to service interface module 183 as yetanother part of function parameters 174. Section 1310 e sets the inputvalues that will be sent to service interface module 183 as part offunction parameters 174. In the illustrated example, section 1310 e isdefined by connection 1210 a in the associated mapping shown in FIG. 11.Section 1310 f calls service interface module 183 using functionparameters 174. Section 1310 g verifies that the call to serviceinterface module 183 was successful. Section 1310 h processes outputdata 169 received back from service interface module 183. In theillustrated example, section 1310 h is defined by connection 1210 c inthe associated mapping shown in FIG. 11.

Although the present invention has been described with severalembodiments, a myriad of changes, variations, alterations,transformations, and modifications may be suggested to one skilled inthe art, and it is intended that the present invention encompass suchchanges, variations, alterations, transformations, and modifications asfall within the scope of the appended claims.

1. A method of transforming data: receiving information defining atransformation of an input data object to an output data object;generating, based on the received information, a script operable whenexecuted to implement the defined transformation; storing the script;receiving a service request from a remote client, wherein the servicerequest requests a particular transformation and identifies a requestdata object; identifying a script that performs the requestedtransformation; generating a response data object by executing theidentified script on the request data object; and transmitting theresponse data object to the remote client.
 2. The method of claim 1,wherein receiving a service request from a remote client comprisesreceiving a service request invoking a web service.
 3. The method ofclaim 1, wherein the service request identifies a request data object byincluding the request data object.
 4. The method of claim 1, whereinreceiving a service request comprises receiving a Simple Object AccessProtocol (SOAP) message from the remote client, and wherein transmittingthe service response to the remote client comprises transmitting a SOAPmessage to the remote client.
 5. The method of claim 1, wherein:generating a script capable of implementing the defined transformationcomprises: generating a service definition describing a request dataobject and a response data object associated with the script; andtransmitting the service definition to the remote client; receiving aservice request comprises receiving a service request that includes arequest data object that conforms with the service definition; andgenerating a response data object comprises generating a response dataobject that conforms with the service definition.
 6. The method of claim5, wherein the service definition comprises a Web Services DescriptionLanguage (WSDL) file.
 7. The method of claim 1, wherein identifying ascript associated with the service request comprises identifying, basedon a name of the request data object, a script associated with theservice request.
 8. The method of claim 1, further comprising storingthe response data object in memory.
 9. The method of claim 1, whereinreceiving information defining a transformation of an input data objectto an output data object comprises receiving, from a user, through agraphical user interface (GUI) information describing an input dataobject associated with the transformation; receiving, from the user,through the GUI information describing an output data object associatedwith the transformation; and receiving, from the user, through the GUIinformation identifying a mapping of components in the input data objectto components in the output data object.
 10. The method of claim 1,wherein the request data object and the response data object compriseextensible Markup Language (XML) data objects.
 11. A method oftransforming data: receiving information defining a transformation of aninput data object to an output data object; generating, based on thereceived information, a script operable when executed to implement thedefined transformation based on the received information; storing thescript; receiving a service request from a remote client, wherein theservice request requests the defined transformation and identifies arequest data object and a requested execution time; generating aresponse data object by executing the script on the request data objectat a time determined based on the requested execution time; and storingthe response data object in memory.
 12. The method of claim 11, whereinreceiving a service request from a remote client comprises receiving aservice request invoking a web service.
 13. The method of claim 11,wherein receiving a service request comprises receiving a Simple ObjectAccess Protocol (SOAP) message from the remote client.
 14. The method ofclaim 11, wherein: generating a script capable of implementing thedefined transformation comprises: generating a service definitiondescribing a request data object and a response data object associatedwith the script; and transmitting the service definition to the remoteclient; and receiving a service request comprises receiving a servicerequest that includes a request data object that conforms with theservice definition.
 15. The method of claim 14, wherein the servicedefinition comprises a Web Services Description Language (WSDL) file.16. The method of claim 11, wherein storing the script comprises storingthe script in a database that includes a plurality of scripts eachoperable when executed to implement a different transformation; andwherein generating the response data object comprises: identifying, inthe database, a script associated with the service request, andgenerating a response data object by executing the identified script onthe request data object.
 17. The method of claim 16, wherein identifyinga script associated with the service request comprises identifying,based on a name of the request data object, a script associated with theservice request.
 18. The method of claim 11, wherein receivinginformation defining a transformation of an input data object to anoutput data object comprises receiving, from a user, through a graphicaluser interface (GUI) information identifying an input data object;receiving, from the user, through the GUI information identifying anoutput data object; and receiving, from the user, through the GUIinformation identifying a mapping of components in the input data objectto components in the output data object.
 19. The method of claim 11,wherein the response data object comprises a database row, and whereinstoring the response data object comprises storing the database row in adatabase table specified by the script.
 20. The method of claim 11,wherein the request data object and the response data object compriseeXtensible Markup Language (XML) data objects.